Non-stochastic Best Arm Identification and Hyperparameter Optimization
نویسندگان
چکیده
Motivated by the task of hyperparameter optimization, we introduce the non-stochastic bestarm identification problem. Within the multiarmed bandit literature, the cumulative regret objective enjoys algorithms and analyses for both the non-stochastic and stochastic settings while to the best of our knowledge, the best-arm identification framework has only been considered in the stochastic setting. We introduce the nonstochastic setting under this framework, identify a known algorithm that is well-suited for this setting, and analyze its behavior. Next, by leveraging the iterative nature of standard machine learning algorithms, we cast hyperparameter optimization as an instance of non-stochastic bestarm identification, and empirically evaluate our proposed algorithm on this task. Our empirical results show that, by allocating more resources to promising hyperparameter settings, we typically achieve comparable test accuracies an order of magnitude faster than baseline methods.
منابع مشابه
Best arm identification in multi-armed bandits with delayed feedback
We propose a generalization of the best arm identification problem in stochastic multiarmed bandits (MAB) to the setting where every pull of an arm is associated with delayed feedback. The delay in feedback increases the effective sample complexity of standard algorithms, but can be offset if we have access to partial feedback received before a pull is completed. We propose a general framework ...
متن کاملUsing Machine Learning to Improve Stochastic Optimization
In many stochastic optimization algorithms there is a hyperparameter that controls how the next sampling distribution is determined from the current data set of samples of the objective function. This hyperparameter controls the exploration/exploitation trade-off of the next sample. Typically heuristic “rules of thumb” are used to set that hyperparameter, e.g., a pre-fixed annealing schedule. W...
متن کاملStochastic Hyperparameter Optimization through Hypernetworks
Machine learning models are often tuned by nesting optimization of model weights inside the optimization of hyperparameters. We give a method to collapse this nested optimization into joint stochastic optimization of weights and hyperparameters. Our process trains a neural network to output approximately optimal weights as a function of hyperparameters. We show that our technique converges to l...
متن کاملCSE 599 i : Online and Adaptive Machine Learning Winter 2018 Lecture 6 : Non - stochastic best arm identification
Example 1. Imagine that we are solving a non-convex optimization problem on some (multivariate) function f using gradient descent. Recall that gradient descent converges to local minima. Because non-convex functions may have multiple minima, we cannot guarantee that gradient descent will converge to the global minimum. To resolve this issue, we will use random restarts, the process of starting ...
متن کاملStochastic Optimization and Machine Learning: Cross-Validation for Cross-Entropy Method
We explore using machine learning techniques to adaptively learn the optimal hyperparameters of a stochastic optimizer as it runs. Specifically, we investigate using multiple importance sampling to weight previously gathered samples of an objective function and combining with cross-validation to update the exploration / exploitation hyperparameter. We employ this on the Cross-Entropy method as ...
متن کامل